A robust nonparametric approach to the analysis of scientific productivity Forthcoming Research Evaluation , April 2003

The paper discusses the potential of recently developed efficiency analysis techniques in the field of scientific research. Several approaches to the study of scientific productivity are compared and their limitations are discussed. As an illustrative example, we analyse data on scientific productivity at (almost) all institutes of the French INSERM and at biomedical research institutes of the Italian CNR for the year 1997. Available data on human capital input and geographical agglomeration allow the estimation and comparison of efficiency measures for the two institutions. The methods applied are nonparametric envelopment techniques (i.e. Data Envelopment Analysis, Free Disposal Hull) and robust nonparametric techniques (based on the concept of order-m frontier) recently introduced in efficiency analysis by Cazals, Florens and Simar (2002). The robust nonparametric approach is applied in this work for the first time for evaluating scientific productivity. It is shown as a useful tool to compute scientific productivity indicators and make institutional comparative analysis. Taking into account a large number of methodological problems, a meaningful and rigorous indirect comparison is made possible. Since the data open a certain number of interpretative problems, we comment on several possible explanations of the observed difference in productivity. 1 The article is a revised edition of a paper presented to the 7 International Science and Technology Indicators Conference, Karlsruhe, September 25-28, 2002. Participants to the Conference provided detailed comments and criticisms. Two anonimous referees offered suggestions that greatly improved the paper. Part of the evidence discussed in this paper had been presented at conferences and seminars at SPRU (NPR Net Conference), CNR (ISPRI, Rome) and INSERM (Institute de Cancerologie, Marseille). We are indebted to Daniele Archibugi, Bianca Maria Potì and Giorgio Sirilli (ISPRI-CNR), Luigi Donato (IFC-CNR), Claude Mawas (INSERM), Gilberto Corbellini (University of Rome La Sapienza) for helpful discussion, although we are the only responsible for the content of this version. We thank Maria Adelaide Caligo (University of Pisa) and Roberto Girardello (Pharmacia) for help in classification of institutes and Marco Brancher and Francesca Pierotti (Sant’Anna School of Advanced Studies) for assistance in data analysis.